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ABSTRACT 



The writing skills of 71 college seniors were assessed at a 
medium-sized public university. Blind scoring, a standard scoring guide, and 
trained graders were used. Curricular writing emphasis was assessed through a 
syllabus study, yielding a curricular emphasis score. The academic records of 
these students revealed the number of courses taken in each discipline, and a 
weighted sum was derived for each student with weighting by the percent of 
syllabi for courses taken that showed "strong emphasis" in writing in each 
discipline. Controlling for entry-level skill in writing, writing score and 
curricular emphasis were highly correlated. Implications for assessment 
practice are discussed. (SLD) 
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Abstract: 



Seniors' writing skills were assessed in 1998 at a medium-sized public university. Blind 
scoring, a standard scoring guide, and trained graders were used. Curricular writing 
emphasis was assessed through a syllabus study, yielding a Curricular Emphasis Score. 
Controlling for entry-level skill in writing, Writing Score and Curricular Emphasis were 
highly correlated. Implications for assessment practice are discussed. 7: ■ 
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Writing Across the Curriculum Works: 

The Impact of Writing Emphasis upon Senior Exit Writing Samples 



Objectives of the Session: The authors have attempted to couple a concern for 
the assessment of writing skills with support for the aims of writing across the 
curriculum. We will try to engage the audience in the innov ative methods that were 
brought together in this research. These methods included those adopted by a university 
writing program for the assessment of students' progress and assessment methods applied 
to the curriculum that students experience on the way to graduation. 

Background: The presentation will review 7 writing across the curriculum and 
some of the difficulties involved in assessing its impact upon students. We will present 
methods adopted in this and similar research to attempt to overcome these difficulties. 

Our methods brought to bear an independent outside assessment of the writing- 
intensiveness of the curriculum. Further, our methods allowed for individualized 
measurement of the writing intensiveness for students’ particular courses of study. 

Outline of Method: A syllabus method was developed by one of the authors and 
presented at a previous VAG conference. The method requires the collection and review 7 
of syllabi for all, or almost all, courses in the undergraduate curriculum. Each syllabus is 
rated by an independent rater on a 3-point scale to measure the degree of emphasis on a 
number of process variables, in this case, writing skill. Specific indicators are used, e.g.. 
number of assigned papers during the term. Evidence of the validity and reliability 
(inter-rater and stability) of this method will be brought out. 

For the writing outcome measure, a sample (N = 71) of the graduating seniors was 
recruited to participate in a senior exit test or writing sample. Students were selected 
from the list of prospective graduates, using a sampling plan designed to guarantee a 
good representation of subjects who transferred in freshman writing course credits and 
those who took those courses at the institution 

The writing sample followed the same procedures and used the same test as was 
used by freshman writing students for the final of the spring term. Grading also followed 
the same structured procedures as used for freshmen. A standard, structured Scoring 
Guide was used and all graders, teachers of freshman English, were trained for grading 
consistency. Scoring was done blind, i.e.. without knowledge of senior vs. freshman 
status. Other data were obtained for statistical purposes. More details can be provided 
•later. 



The academic records of these students revealed the number of courses taken in 
each discipline. A weighted sum for each student, with weighting by the percent of 
syllabi that showed “strong emphasis” on writing in each discipline from the syllabus 
study, yielded a Curricular Emphasis Score. 

Outline of Results: Correlations among all variables were examined with a view 
toward predicting skill in writing. For summary purposes, the Total Score will be used in 
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this proposal in place of the other six writing outcome variables, of w hich it is the 
composite. Total Score correlated significantly with freshman English grades (.43). 
cumulative GPA (.38). and SAT-Verbal (.57). In addition, freshman English grades 
correlated .58 with the cumulative GPA. All correlations w r ere highly significant. 

The role of transfer was also examined. The differences between participants 
who transferred their freshman English credits from another institution and those who 
took their English at the university were not statistically significant. These results were 
consistent with those found in a 1997 writing study. 

For 25 students randomly selected to derive a Curricular Emphasis Score, the 
correlation between that score and the Total Score on the writing test was r = .40. p < .05. 
To control for entry-level skill in writing a partial correlation w r as conducted on 
Curricular Emphasis and Total Score on the Writing Test, using SAT-Verbal scores as 
the controlled variable. The partial r was .78, p < .00 1 . 

Major Conclusion: The real message is that of the title of our presentation: 
“Writing Across the Curriculum Works.” We believe our study has overcome at least 
some of the obstacles that stand in the way of making such a claim with confidence, if not 
certainty. Professors of English as well as many others, who believe in and care deeply 
about fostering writing ability in college, and who have labored toward this end for many 
years, can be encouraged by these results. 

The authors bring to the presentation a rare combination of assessment 
perspectives and a professional focus on the improvement of writing. The cross- 
fertilization of ideas, leading to several innovative assessment methods, should be 
stimulating to most conference attendees. Both the findings and the methods should have 
applicability across most institutions. 
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Method 




© Syllabi were reviewed for evidence of seven ’process' variables’ 

© written communication, oral communication, problem-solving, computer applications. 

mathematical applications, international perspectives, and diverse perspectives 
© Scored on a 3-point scale 

© No Emphasis - no evidence that the process variable was being addressed in the 
course (which does not necessarily mean that it was not - just that the syllabus didn’t 
reflect it) 

© Some Emphasis - some evidence that the process variable received some attention at 
some point in the semester (e.g., a lecture on an international topic, one writing 
assignment, an emphasis on oral participation in class) 

© Strong Emphasis - evidence that the process variable received emphasis throughout 
the semester (e.g., International Finance, Psychology of Sex and Gender, an 
accounting course with problem-solving throughout the semester) 

■ for written communication, strong emphasis was defined as two or more papers 

■ for oral communication, strong emphasis was defined as an oral presentation to 
the class 
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Results of the 1 999 Syllabus Study 
Christopher Newport University 
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From 528 syllabi in 1999. 
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Comparison of the 1996 and 1999 Syllabus Studies 
Christopher Newport University 




From 407 syllabi in 1996 and 528 syllabi in 1999; 
All Chi Squares < 2. non-significant differences. 
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Conclusions 



* - ijt *-^7 ~4 amaa 



® Curricular emphasis on these seven process variables has not 
changed significantly between 1996 and 1999 
© at least as reflected in the syllabi 
® Two-thirds of CNU courses incorporate written communication and 
oral communication 

© about one-fourth incorporate two or more writing assignments 
and about one-fourth incorporate oral presentations 
® More than 80% incorporate active problem-solving 
® About one-third incorporate computer applications and 

mathematical applications, and one fourth incorporate international 
perspectives 
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Method 



® In the spring of 1998, writing samples were taken from 71 seniors using the 
same writing prompt 

© similar data were available from samples of seniors from 1995 and 
1997 

© (seniors in the 1997 sample had higher GPAs and SAT Verbal scores, 
and perhaps because of this, tended to score higher on the writing 
samples) 

® Essays were scored by several faculty members from the Department of 
English, using a 5-point scale 
© 0 = failing 
© 1 = below average 
© 2 = average 
© 3 = above average 
© 4 = superior 
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Dimensions of Scoring Guide 


Dimension: 


Definition: 


Summary of reading (Content) 


Understands Author's Central Thesis and 
Purpose 

Covers Main Points Concisely 
Accurately Represents Original 


Critique of Reading (Content) 


Evaluates the Author’s Central Thesis and 
Purpose 

Critiques the Main Points of the Argument 
Evaluates the Strategies of the Author 


Personal Response to the Reading 
(Content) 


Develops a Thesis that Guides the 
Response 

Supports the Thesis with Examples and 
Evidence 

Draws Inferences and Makes Connections 
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Dimensions of Scoring Guide 

Dimension: Definition: 



Structure Introduces Content with a Sense of 

Purpose and Focus 
Builds Orderly, Developed Paragraphs 
, Establishes Coherence with Logical 
Transitions 

Concludes with a Sense of Focus and 
Meaning 

Correctness Employs Quotations from the Readings 

Correctly 

Demonstrates Understanding of Grammar 
Uses Conventional Punctuation 
Demonstrates an Understanding of 
Mechanics and Spelling 

Style Draws from Varied and Complex 

Vocabulary 

Varies Sentence Patterns 
Exhibits Clarity and Concision 
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Results from the 1995, 1997 and 1998 Studies of 
Writing 

Christopher Newport University 

lit lh 



■ 1995 ■ 1997 □ 1998] 




Summary Critique Personal Structure Correct Style Total 



From 60 seniors in 1995, 93 seniors in 1997, and 71 seniors in 1998. 
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Predicting Skill in Writing 
Correlation Matrix 






Freshman ENGL Cum GPA 
Grade# 


SAT-Vertoal 


Total Score 


Freshman ENGL 1.00 
Grade# 






Cum GPA .58* 1.00 






SAT Verbal .27 .24 


1.00 




Total Score .43* .38* 

r 


.57* 


1.00 


•Correlations significant at the .01 level; 

these correlations are very similar to those found for the 1997 sample. 
#Average of two Freshman English course grades. 
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Correlations between Mean ENGL Grade and GPA 
for Students who had Transferred ENGL and 
Students who had taken ENGL at CNU 





Transfer ENGL# 


CNU ENGL# 


CNU GPA 


.37* 


.54* 


Transfer GPA 


.62* 


.47 


Cumulative GPA 


.59* 


.54* 


•Correlations significant at the .01 level; 

#Average of two Freshman English course grades. 
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Comparison of Transfer ENGL and CNU ENGL 



Freshman ENGL - Total Writing Score 
Grade# 



Transfer ENGL M = 2.77 M = 2.01 



CNU ENGL M = 2.55 M = 1.85 



As was true for the 1997 writing sample, these differences are not statistically significant. 
#Average of two Freshman English course grades. Means for TNCC transfer students are 2.80 
for ENGL grade and 1.87 for Total Writing Score 
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Comparison of Transfer and Native Students 

■ ■ , -- 1 I. 

Freshman ENGL Total Writing Score 

Grade# 



Transfer Students M = 2.74 M = 1.96 



Native Students M = 2.52 M = 1.95 



As was true for the 1997 writing sample, these differences are not statistically significant. 
#Average of two Freshman English course grades. Native students had no transfer credits. 
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Curricular Emphasis and Skill in Writing 

I —- — — 

® For a sample of 25 of these students, a Curricular Emphasis Score 
was derived by determining the number of courses taken in each 
discipline and weighting by the percent of. syllabi that showed 
Strong Emphasis on Writing in each discipline (from the Syllabus 
Study) 

® This Curricular Emphasis Score was then correlated with the Total 
Score on the Writing Test for these 25 students 
© r = .40. p = .05 

® To control for entry-level skill in Writing, a Partial Correlation was 
conducted on Curricular Emphasis and Total Score on the Writing 
Test, using SAT-Verbal scores as the controlled variable 
© r = .78, p = .001 
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Conclusions 



® Mean grade in Freshman English courses, cumulative GPA, and 
SAT-Verbal scores all correlate significantly with an independent 
assessment of writing skills of seniors 
® There are no significant differences on this senior assessment of 
writing skills between native and transfer students 
© or between students who had taken Freshman English at CNU or, 
elsewhere 

® Curricular emphasis on writing correlated significantly with this 
senior assessment of writing skills 

© this correlation was even stronger after controlling for differences in 
entry-level writing skills as measured by SAT-Verbal scores 
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